The present invention relates generally to the field of wireless devices and, more particularly, presentation and display of content on such devices.
Until recently, applications for mobile devices were text based with limited interactivity. However, as more wireless devices are equipped with color displays and more advanced graphics-rendering libraries, consumers will demand a rich media experience from all their wireless applications. Rich media content generally includes content that is graphically rich and contains compound (or multiple) media including graphics, text, video and audio and, preferably, delivered through a single interface. Rich media dynamically changes over time and could respond to user interaction. Rich media applications, particularly in the Web services domain, include XML-based content such as Scalable Vector Graphics (SVG), Synchronized Multimedia Integration Language (SMIL) and Compound Documents Format (CDF).
SVG (more specifically, version SVGT 1.2) is a language for describing two-dimensional graphics in XML. SVG allows for three types of graphic objects: vector graphic shapes (e.g., paths consisting of straight lines and curves), multimedia (e.g., raster images, video), and text. SVG drawings can be interactive (e.g., using a Document Object Model, or DOM, event model) and dynamic. Animations can be defined and triggered either declaratively (e.g., by embedding SVG animation elements in SVG content) or via scripting. Sophisticated applications of SVG are possible by use of a supplemental scripting language which accesses the SVG Micro Document Object Model (uDOM), which provides complete access to all elements, attributes and properties. A rich set of event handlers can be assigned to any SVG graphical object. Because of its compatibility and leveraging of other Web standards (such as CDF, described below), features like scripting can be done on XHTML and SVG elements simultaneously within the same Web page.
SMIL (specifically, SMIL 2.0) enables simple authoring of interactive audiovisual presentations. SMIL is typically used for rich media/multimedia presentations which integrate streaming audio and video with images, text or any other media type.
The CDF working group is producing recommendations on combining separate component languages (e.g., XML-based languages, elements and attributes from separate vocabularies), like XHTML, SVG, MathML, and SMIL, with a focus on user interface markups. When combining user interface markups, specific problems have to be resolved that are not addressed by the individual markups specifications, such as the propagation of events across markups, the combination of rendering or the user interaction model with a combined document. The CDF working group will address these types of problems.
In order to maximize the user experience while viewing rich media content, efficient delivery of such content of the client is important. However, optimal content presentation is also vital and is more challenging when viewing orientations have to be changed, either explicitly (e.g., declared in the content itself) or implicitly (e.g., via a change in the actual phone screen mode).
A growing number of mobile devices (e.g., Nokia N93, N95) have greater versatility in their form factors, particularly screen orientation and input modalities. While the presentation of rich media content on a client device is scalable and interactive, challenges still exist in presentation when conforming to changes in screen orientation. There is hence a need for the content or the layout of the content displayed on the screen to adapt to these orientation modes for delivering a good user experience.
Typically, built-in advanced sensors in the phone such as an accelerometer, detect when the phone's screen is rotated and, in turn, generate events that notify the top-layer application framework. This notification enables the application to respond to the orientation events to perform certain actions such as changing the appearance of the application (e.g., the application grid on the phone screen) or the content within the application by dispatching this event to the engine that is responsible for rendering or displaying the content. As an alternative, these orientation events may be explicitly streamed from a remote terminal to the client and executed with or without event triggers. However, in order for the underlying engine (for e.g. rich media engine) to understand these application events, it is required to provide mechanisms for enabling proper response to screen orientation events including:                a syntactic definition of the screen orientation modes in the rich media content;        a functional and syntactic definition of a rich media based event interface specific to these screen orientation modes; and        a methodology on how phone hardware based screen orientation events map to the application based event interface, the subsequent processing of these events and actual content realization by the rich media client engine.        
Although the concept of different screen orientations has been prevalent for various types of user interface (UI) systems and devices, currently there is no systematic mechanism that defines the handling of screen orientation modes in the context of a rich media environment.
A preliminary contribution was submitted at a Third Generation Partnership Project (3GPP) SA4#41 meeting (ref: ftp://ftp.3gpp.org/TSG_SA/WG4_CODEC/TSGS4_41/Docs/S4-060640.zip) to address this problem. The proposal introduced four events based on angles:
Event IdentifierNamespaceDescription“screenOrientation0”urn:mpeg:mpeg4:laser:2005The screen orientation haschanged to typical ‘landscape’orientation“screenOrientation90”urn:mpeg:mpeg4:laser:2005The screen orientation haschanged to typical ‘portrait’orientation“screenOrientation180”urn:mpeg:mpeg4:laser:2005The screen orientation haschanged to inverted ‘landscape’orientation“screenOrientation270”urn:mpeg:mpeg4:laser:2005The screen orientation haschanged to inverted ‘portrait’orientation
However, with this proposal, the event identifiers are based on angles and, therefore, introduce ambiguity while mapping the underlying hardware/OS based events to these content based events. For example, the Symbian UI framework does not support inverted portrait and landscape modes. Further, the proposal does not define the mapping of the event identifiers to an event interface. In addition, the proposal lacks the definition of event interface, which is crucial for obtaining information related to the screen orientation, such as screen width/height, and the location of the soft keys in response to the change in screen orientation.
Further, U.S. Patent Publication No. 2002/0101439, titled “Method and Apparatus for Rotating an Image on a Display,” utilizes a three-dimensional rendering engine to rotate an image based on a user-selected or otherwise determined screen orientation. A vertex coordinate transformation is defined for a rotated destination image. The source image is used as a texture for texture mapping during rendering operation to produce rotated image. However, this disclosure mainly concerns image manipulation via rotation and texture mapping to align with the desired screen orientation.
U.S. Pat. No. 6,597,384, titled, “Automatic Reorienting of Screen Orientation Using Touch Sensitive System,” discloses the automatic change of display by sensing a change in the direction of the display via touch sensors. This disclosure focuses primarily on the mapping of hardware events triggered by touch sensors to the actual orientation of the screen. It fails to address any application level functionality for handling such orientation and is restricted mainly to touch screens.
U.S. Pat. No. 5,910,797, titled, “Portable Data Processing Apparatus Provided with a Screen and a Gravitation-Controlled Sensor for Screen Orientation,” discloses a gravitation-controlled sensor in the screen for measuring a spatial orientation. The disclosed apparatus has a programmed data processor under control of a predetermined range of spatial orientations imparting a non-stationary motion pattern to a predetermined selection among the objects. The motion can be used in the way of a joystick. This disclosure is related to hardware mechanisms for enabling screen orientation, rather than any application-level functionality, as needed for rich media content.
U.S. Pat. No. 5,661,632, titled, “Hand Held Computer with Dual Display Screen Orientation Capability Controlled by Toggle Switches Having First and Second Non-Momentary Positions,” discloses an architecture of a handheld computer having a generally rectangular housing on a front side wall of which a display screen is operatively mounted. A row of toggle switches is also mounted on the front housing side wall, adjacent the display screen, the toggle switches being operatively connected to the computer circuitry within the housing. One of the toggle switches is operative, via the internal computer circuitry, to selectively rotate, through an angle of 90 degrees, the orientation of data generated on the screen so that the screen data is in an upright viewing orientation relative to the user of the computer. Because the toggle switches are non-momentary, the orientation of the data generated on the screen is preserved while the handheld computer is turned off. Again, this disclosure concerns hardware rather than application level mechanisms for screen orientation.